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This document presents two papers given at a 1996 conference 
on children's mental health. The first paper is "New Approaches to Evaluating 
Systems" by Heather B. Weiss. This paper discusses the need for evaluations 
of systems change through the development and use of evaluation databases of 
both comprehensive programs and system reform efforts. It emphasizes the need 
to sustain and scale up successful systems change initiatives, the need to 
reinvent evaluation to build accountability into the resource allocation 
process and demonstrate that systems reform is productive and better than 
alternative approaches. Suggestions for building a system of accountability 
are offered. The second paper is "Outcome Accountability and System Reform: 
What Should They Mean in Policy and Practice?" by Charles Bruner. This paper 
considers three themes in the current move to outcome -based accountability in 
systems reform: the current political demand for improved outcomes, the need 
to change current responses to improve outcomes, and the need to set 
evaluation parameters. (Individual papers contain references.) (DB) 
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We are all struggling with systems change and how to create knowledge development strategies that will 
inform our efforts to develop better systems of care for children and families. It is important, now more 
than ever, that we learn from each other in terms of reinventing evaluation. I propose that we follow a 
model of co-learning &shyp; the co-leamers in this instance being those who are trying to effect change 
and those of us who are evaluating them, or as I like to think, informing them as they do their work and 
develop reflective practice. I would like to make three observations about co-learning. 

First, with this "co-learning model" comes a different relationship with the people we serve; it is no 
longer an expert and a subject, or the researcher and the researched. We cannot underestimate the 
difficulty of becoming co-leamers, or the change that it means for all of us, at both ends of the co-leamer 
relationship. 

Second, we need to reinvent evaluation, or knowledge development, so it becomes an internal part of 
doing business in our large systems. Evaluation needs to be part of the process to illuminate support and 
assess our actions as we craft our services for children and families. When it is internalized, it is no 
longer out there being done to us, but instead is in here being done with us. 

Third, we have to recognize the partner who has always been there, and that is the public. We are in an 
era, which I think of as "the new social policy," in which public accountability has come to the forefront. 
So, another constituency or stakeholder in the co-learning process is the public, and we need to reflect on 
the "public will" that we must generate to maintain the service structure, redesigned or not. 

My colleagues and I are learning from actual evaluations of systems change and from the risk takers who 
are trying different evaluations strategies. In addition, we just started a new inquiry for understanding 
process studies. The convention is that you cannot undertake an evaluation without doing a process 
study. So we have been using our evaluation databases of both comprehensive programs and system 
reform efforts to understand what people mean when they talk about process evaluation. 
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Creating Systems Change 

We have identified some of the earmarks of systems change, the contours of emerging practice, and the 
challenges that come with efforts to do evaluations differently. I want to argue that we cannot do 
systems change unless we also build in the knowledge development strategies that inform change, as 
well as assess it. 

In the child and family arena, we have said that simple programs and categorical approaches are not 
enough and that we need systems change. We have said that we have to develop systems that provide a 
set of services, preventive and otherwise, to children and families. So across the board, we bought the 
notion of systems reform, in part because we know what we have does not work &shyp; not because we 
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know what the shape of the system reform looks like. 

That puts us in a particularly difficult position because we are inventing the system as we implement it. 
Systems change involves many actors and reconfiguring so many different things, that we are talking 
about creating a very, very complex entity. 

We know that our comprehensive programs have not worked, not least because we cannot scale them up. 
We have spent the last ten years watching the family support arena evolve from a fairly simple set of 
programs into more comprehensive systems. This has led to a growing realization the family support 
initiatives are context-dependent, that comprehensive efforts require linkage to other services families 
need, and that no single agency can adequately honor a holistic approach to families. We know we have 
to build linkages, collaboration, and coordination. 

Our landscape, however, is full of programs and some systems change initiatives, that demonstrated, 
researched, and died. The challenged, then, is to sustain and scale up these initiatives &shyp; something 
we do not yet know how to do. These efforts bring us instantly into a political context that has to be 
honored-again, this comes back to the issue of "public will." We must create knowledge and 
accountability that will warrant public faith in what we are trying to do. 
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The Need to Reinvent Evaluation 

We can start by building our case on evaluations of CASSP and, other, usually single system, reform 
initiatives that suggest systems reform is possible. Currently, we are at the same point in evaluation as 
we are in systems reform. We need to recreate and rethink what we do, but it is not clear what the path 
ahead looks like. Change will require a willingness to take risks, and feedback through a co-learning 
model so we can revise our evaluation practices, as well as our program practices. Hence, we have to 
reinvent evaluation and knowledge development, so that they become features of systems reform that 
work hand-in-hand with the effort. 

Systems reform is difficult, and requires much of evaluators. The goals of system reform include the 
major restructuring of existing services. We need to rethink and redirect public resources and public 
services into a coherent system. This cross-system effort should look at the needs of an entire 
community, not simply particular population groups. It also should involve service strategies that are 
different, those that go beyond the coordination of existing services and create a new system that is 
integrated and user-friendly. It should be a service system that operates differently, and looks and feels 
different to families who are using the system. We must involve multiple stakeholders in these efforts, 
including program participants, front-line workers, managers, and those who decide how resources are 
allocated in communities. 

Additionally, we need to build accountability into the resource allocation process. This means creating 
responsive governance entities that can manage accountability and design different financing strategies. 
We cannot underestimate the risk, complexity, and challenge that accountability poses with multiple 
stakeholders at the state and local level in systems reform. 

We need to demonstrate that systems reform is productive, that it can be implemented, that it produces 
important outcomes, and that it is better than alternative approaches to organizing and delivering 
services. With this comes a need for internal validity, high quality evidence, and external validity, so that 
this evidence is useful not only to evaluators, but to other stakeholders as well. We need to inform 
program development, build and sustain the public will, and enlist a high degree of participation to make 
the overall system work. 

These are enormous challenges to address, particularly in the context of the new social policy. Under the 
new Congress we are going to have more and more decentralization, more and more block granting, and 
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fewer and fewer entitlements. We are going to have a very different look in states and communities 
around the country. There are new players and new stresses. I think there will be incredible competition 
for increasingly scarce resources. 
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What Will Change Mean? 

What does this mean for those of us who have been laboring in the vineyards of systems reform? I think 
it means we have to redouble our efforts and confront a number of challenges. A couple of years ago the 
governor of West Virginia set up a new cabinet for children, youth, and families that aimed to 
decentralize planning and decision making about the use of resources for children and families in the 
state. The cabinet gave money to a number of communities around the state and asked an array of 
players to participate. They were told that they would have control of much of the existing revenue, and 
they had to plan a new system which met everybody's needs, underscored prevention, and also made it 
possible to access the kind of treatment that some families would need. 

These communities had to plan, implement, and assess the effectiveness of their efforts. They were 
scared to death and very much felt they did not have the kind of information they needed. They went out 
and collected some demographic data, but realized that it was necessary, but not sufficient. They then 
started doing surveys, focus groups, and so forth, to get the input they needed for planning this new 
system of services. When it was time to evaluate their efforts, they were terrified. They could not find an 
evaluator who would come in, especially with their very scarce resources, to help them perform a 
participatory evaluation. 

This, I think, is the reality in many, many communities. We decentralize and say that we are 
empowering communities, but we give them disempowering evaluations and fail to make them 
stakeholders in the entire process. I think we need to come to grips with this as we reinvent evaluations. 

What are some of the contours of the new evaluation practice? We need to move beyond experimental 
research design. Much of our practice, particularly as it has been played out at the highest levels of 
government, says that there is no knowledge past random assignment. We need to build a case for 
developing other methods that are stringent and that provide useful knowledge, above and beyond what 
can be achieved through an experimental research design. There are questions about the ability to 
implement, pure and simple, an experimental research design in systems reform efforts. Usually when 
one does experimental research design, it is with a program. 

There is a specified treatment, an external evaluator. From this we determine whether something works 
or not&shyp;usually not "how" something works. In experimental research designs, we make a whole 
set of assumptions about research validity and "treatment," which I think are not possible to guarantee in 
systems reform efforts. 

As I have looked across the terrain in the 20 years that I have studied programs for children and families, 
I have not seen any examples in which a well-done evaluation, meeting the conventions of experimental 
research design, has resulted in any kind of significant program expansion. I have increasingly come to 
think that, in the social services arena, these evaluations create reasons why we don't do things, as well 
as inform what we do. 

We need to ask what we need to prove, to make a plausible case, that some new initiative is working. 

We need to identify alternatives to experimental research design, and ask the question at the community 
and state level, "who are the stakeholders, what is proof, and what's a plausible case that says that 
systems reform works?" 

People like me argue that we need alternatives to conventional practice. In fairness, I am asking you to 
open up a Pandora's box of problems. The more we look at alternative practices, the more I see comfort 
in the traditional recipe for doing conventional research using experimental design with an experimental 
evaluator who is the expert, assumes control, and so on. What does evaluation as a helping relationship 
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really look like? It means that the role of the evaluator is fundamentally different, and it means a lot of 
stress and tension as we try to redefine this helping relationship &shyp; the co-learning model, if you 
will. It means we are in a context with different stakeholders who have different expectations for what 
we are going to do. Part of the co-learning model is thinking through those different expectations and 
somehow working out a reasonable way of proceeding in the face of them. 

I know it is not "going native" as they say in anthropology, and becoming completely the voice of the 
program. I know it is not being remote and disconnected. The answer lies out in the middle, in 
unexplored territory, particularly against the backdrop of a group of stakeholders who have different 
ideas about objectivity, bias, and the rest of it. 
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Building a System of Accountability 

What does it mean to build a system of accountability? It is very different than building a good 
evaluation and knowledge development strategy. Accountability is when you learn and see how you are 
doing. You also generate stakeholder involvement so there will be an audience who will listen to what 
you have found, who will then have the responsibility for problem solving, and when necessary, pulling 
the plug when something is not working. This is a different game than evaluators have been in before. 
There is an emphasis on problem solving and midcourse correction, but there also the notion of "plug 
pulling" that we may not have faced in the way we will need to in this new social policy environment. It 
also means figuring out the rewards for success for risk takers. 

We also need to understand the process of developing systems change initiatives. Some of the work by 
the late Judy Gardner and her colleagues at the Heller School at Brandeis suggests that systems reform 
efforts build on one another &shyp; that CASSP created the infrastructure on which the Robert Wood 
Johnson Mental Health Services for Youth Initiative was able to build. We need to get the historic 
dimension, and get a sense of the infrastructure of systems reform, document that, and extract the 
lessons for others. One of those lessons may be that systems change takes time. 

With this new understanding of process comes the notion of mixed methods, and getting beyond what 
we call anecdotes for "correlation coefficients." We've been looking at process studies and are amazed at 
how many of the process studies embedded within evaluations are about "parallel play." The correlation 
coefficient is over here, and the case study that demonstrates that is over there. There is really a need to 
connect these two methods so they work together, complement one another, and produce knowledge that 
cannot be generated from a purely quantitative or a purely qualitative strategy. 

We need to spend more time in training and capacity building for evaluators. One of the things we hear 
is how few evaluators there are who have the negotiation and other skills necessary to work in these new 
ways, in this co-learning model. As evaluators, we need to examine how we are training people in these 
skills that will be requisite, and how we can get this training in place. It may require developing 
apprenticeships with innovative evaluators that enable the next generation to learn from the experience 
and the struggles of this first generation of risk takers. 

There are a set of earmarks for this new framework for evaluation. First, this new strategy must develop 
the internal capacity to collect, analyze, interpret and utilize a wide variety of data. It must incorporate 
knowledge development as one of the central functions of systems reform itself. 

Secondly, the work is never done. When we develop a system, it has to be one that is continually 
reinventing itself as circumstances change and as the needs of children and families change. This 
capacity for continuous self-reflection an important element. We must use information to inform a wide 
variety of stakeholders, and build consensus about the outcomes we want and the strategies we will use 
to improve those outcomes. Finally, we must use information to build and enforce a sense of public 
accountability for this process. 

When we looked at examples of people who have been trying to do these strategies, we found four types: 
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• Mixed methods strategies, in which people are using quantitative data and simultaneously talking 
to communities and trying to get participant perspectives on implementation and the value of 
programs; 

• Knowledge development and self-evaluation strategies, which Bob Illback has done in Kentucky, 
and which Lynn Usher has done in the family resource centers in Maryland, as part of the 
Maryland Service Reform Initiatives; 

• Public accountability, which is the outcome-based accountability strand, which you see a great 
deal of, particularly in the United Way Success by Six initiatives; and finally, 

• More participatory strategies which are more in line with some of things I've been suggesting. 

We have learned several lessons from trying these new approaches. One is that it is incredibly difficult 
to promote organizational learning and the capacity for reflective practice. Those difficulties lie both in 
our stance as evaluators and in the receptivity and structure of the organizations we're asking to become 
learner organizations. There's a recent article in Evaluation Practice that talks about how few instances 
there are where organizations have really learned from evaluation. A profound change in organizational 
culture is necessary if we are going to implement these new evaluation approaches. 

We've also found that it is a real struggle getting trained evaluators for this organizational development 
approach to evaluation. They are simply not there, and when they are there, they require time and 
resources, including money, to implement this kind of practice. More self-evaluative, participatory 
approaches mean time on the ground, working with front line workers and managers who are collecting 
data, asking them about how they use that information to inform practice, to inform the multiple 
stakeholders in the community, and to be accountable. We cannot underestimate the time and resources 
that are necessary to make this work. 

I would argue, however, that if we do not open the Pandora's box of these new frameworks for practice, 
we will be stuck in a situation where there are fewer and fewer resources and more public distrust of our 
efforts. Our evaluation project at Harvard Family Research Project is trying to create a broader forum for 
our risk takers. We have launched our newsletter, The Evaluation Exchange, which is meant to be a 
place where people who are trying new approaches can get some visibility for their work and get 
feedback from what we hope will be a growing leadership. We are embarking together down this road of 
trying new strategies, and we need to be co-leamers in sessions such as this, confronting challenges of 
inventing and implementing this new of practice. 
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Today I am going to discuss with you three themes in the current move to outcome-based accountability 
in systems reform: The first is the current political demand for improved outcomes; the second is the 
way we need to change our responses if we are going to improve outcomes, and third, is the way we 
must set parameters upon which to judge our efforts. 

The Demand for Improved Outcomes 



In today's political climate, policy makers have discovered outcomes and they want to use them on us. 
There is increasing recognition that too many children and families are not succeeding and growing 
rhetorical acceptance that the characteristics of the current service systems&shyp;categorical, reactive, 
fragmented, arbitrary, hierarchical, deficit based, coercive &shyp;do not provide a good match between 
what families need to succeed and what systems offer. 

The growing use of today's new service mantra&shyp;"family-centered, neighborhood-based, 
community-designed, preventive, partnering, flexible, comprehensive, asset oriented, 
voluntary"&shyp;suggests how services have to change to provide a better match and what is needed to 
succeed. 

Belief in this new service mantra requires new accountability that is not based upon adherence to 
procedures but rather upon success in attaining results. 

Increasingly, outcomes are seen as an integral part, and in many respects, a driver of system reform. 
System reform can be conceptualized as requiring integration of new forms of governance, new 
approaches to services, new financing systems, attention to leadership and professional development, 
and clear enumeration of outcomes and means to hold systems accountable to meeting them. 

At the state and community elected official policy making level, officials want something "that works" 
to reduce teen pregnancy, infant mortality and low birthweight, school unreadiness, crime and juvenile 
delinquency, and low skills, school dropout, and idleness. These officials are not necessarily naive, but 
they want to put the pressure on bureaucracies and service providers for solutions and for fixing what 
advocates and reformers say is broken. "You want flexibility, we want results." 

Return to Top 



Changing our Response 

If we are going to improve outcomes (and to succeed with children and families current systems will 
fail), we need to change our responses at several levels. I've proposed a conceptual framework to view 
needed comprehensive service reforms. 
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The "simple model" depicted in Figure I g enerally produces a constellation of outcomes that are 
multi-dimensional, spanning health, developmental, social, and psychological dimensions. 

The charge to implementers is to improve all these outcomes at a community-wide level. This requires 
change, particularly as it is related to eliminating the categorical deficits of the old system and replacing 
them with the service mantra of the new. 

Some of the positive outcomes for children, which are generally consistent with what community or 
state stakeholders want to produce include: 

• Success in school 

• Responsible sexuality 

• Social consciousness 

• Community involvement 

• Healthy lifestyle 

• Readiness for adulthood 

Policy makers often stop here and turn the responsibilty for improved outcomes over to implementers. 
These can be considered an interrelated constellation not amenable to fragmented, categorical responses. 
These outcome indicators are adapted from Lisbeth Schorr's "minimalist list." 

What we need to ask is: 'Can these long-term outcomes (the British use the term "final" outcomes) be 
used to guide reform?' 'Can they be used to hold systems accountable?' 'Can they inform how different 
aspects of our needed and complicated array of services and supports must change? 

The answer is yes and no. They can be used as a guide to creating pressure for reform. At the same time, 
we need different, intermediary or proximate, levels of outcomes or outputs to use for accountability and 
course correction purposes, applicable in different ways to different parts of the system of community 
supports and services. 

I've proposed a formulation of three conceptually distinct, but interrelated, outcomes or outputs needed 
to expect changes in those long-term goals: the whole village approach, effective frontline strategies, and 
productive service systems. I will describe the importance of each, and then touch on ways that one can 
measure them. 

Whole Villages. The most common frontispiece to a task force on systems change through 
comprehensive services is the African proverb, "It takes a whole village to raise a child." Note that it 
does not say that "it takes a multidisciplinary team with integrated case management to raise a child." 

In distressed and disinvested neighborhoods, however, there often is a threadbare community fabric 
offering little support and opportunity to help families grow and develop. The challenge raised by the 
proverb is really, "How do you make whole villages, or make existing villages whole?" 

There exists substantial empirical literature that living in distressed and disinvested neighborhoods 
constitutes a major risk factor to healthy growth and development. Various formulations incorporating 
these village factors into reform strategies have been offered: Cataldo and Hawkins and others have 
marketed them as "protective factors;" Chapin Hall speaks of them as "primary services;" Putnam and 
others describe this as "social capital;" and Public/Private Ventures speak to them as "core concepts." 

One enumeration of these community elements or factors in child success can be described as: 

• Social and economic stability, 

• Safe and stable neighborhoods, 

• Nurturing and mentoring adults, 

• Positive peer attitudes, 

• Diversity of "social capital" &shyp; school, church, neighborhood, civic activities, and 

• Opportunities to realize aspirations. 

Does this list make sense? Clearly, where these do not exist in a neighborhood (whatever other services 
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are provided), children are not likely to succeed. Where they do exist, most children do pretty well. 

Developing strategies to construct such core elements represents a challenge that must be met if the final 
outcomes are to be achieved. We're in our infancy of exploring how to do that. At the policy level, recent 
proposals to offer Midnight basketball to deter juvenile crime is a good illustration. 

While we may not have answers on how government can be a catalyst for the creation of "Caring 
communities", we can develop metrics that do a reasonable job of measuring each of these conditions. 
Some approaches, most notably family support centers and settlement houses, seek to construct this 
village as well as work with individual families. There are outcome measures or indicators that can be 
tracked to determine whether communities are rebuilding, as this more detailed iteration of whole village 
concept for to child and youth suggests. 

Effective frontline strategies. The second change that needs to occur is in frontline service delivery. 
How do we measure, and hold accountable, this new service mantra frontline approach that is 
individualized in its response to families, given that families are messy units of analysis and do not fit 
comfortably within regression equations? 

First, we should have process measures to determine whether services actually are community-based, 
individually-tailored, asset-oriented, and comprehensive, which in itself is a major task. I talked with a 
collaborative leader enamored of Lisbeth Schorr's book, Within Our Reach, whose solution was to beat 
workers over the head with it until they agreed to practice exactly what she said. This service mantra is 
easier to enact into policy and agency mission than it is to create in practice. 

With outcomes, however, I believe we must develop means to determine whether individual workers are 
making progress with families. Without this, broader accountability will be impossible&shyp;with it, 
that accountability may be complex but is at least technically possible. The following characteristics of 
effective front line interventions adapted from my paper in the National Center for Service Intergration 
Brief, Making a Difference, suggests different levels of examining such service approach outcomes at 
the frontline level: 

• Identity Appropriate Families (Service Penetration), 

• Engage Families, 

• Aid in Family Growth, 

• Connect to Community-Building, and 

• Produce Changes in Other Systems. 

This can provide a discipline for programs involved in this admittedly "mushy" family support/frontline 
practice. This is critical to do, because the initial gains with families may be quite removed from 
long-term community outcomes. 

For example, let me illustrate some of the steps families in Iowa's Way to Grow program took and 
contrast them with the community outcomes that funders attached to the program. This comparison 
indicates that the relationship between what steps families take and what long term outcomes policy 
makers seek usually is not linear or direct (See Table IT 

Productive Service Systems. The third issue in developing positive outcomes addresses systemic 
responses-&shyp;raising issues of collaboration, referral, and integrated activities. We still need 
expertise and professional service responses-&shyp;but we need them to practice differently and connect 
better with one another. 

Suggested outcomes we want to see from changes in the manner in which our systems respond to 
families might include: 

• Seamless and coordinated response, 

• Integrated into community life, 

• Timely response to opportunities. 




10 



12/14/98 11:47 AM 



http://lumpy.fmhi.usf.edu/CFSroot/rtc/proceeding8th/8th.417.html 



• Sufficient to meet objectives, 

• Efficient in meeting objectives, and 

• Adaptive and learning objectives. 

Again, these are based upon common sense notions of where our current systems often fail to respond 
adequately and appropriately. They are based upon what we believe, from both theoretical and clinical 
roots, systems need to do to respond productively to children and families. 

These system outcomes or outputs can be measured as well&shyp;often even at a performance tracking 
and management information system level. In child welfare, for instance, the implications of such an 
approach to permanency planning would be that a premium would be placed on more immediate 
response, minimizing disruption and retaining connected with family the these when placement occurs, 
and establishing the expectations for everyone in the system to achieve permanency. This is in dramatic 
contrast to the usual business in child welfare of placing a child, breathing a sigh of relief, and beginning 
the process of assessing reunification potential only as the court review date approaches. 

There is nothing terribly profound in all this. It seems to me that positive final community outcomes 
logically must be dependent upon whole villages, effective frontline approaches, and productive service 
systems. The absence of any diminishes the ability of all to reach those outcomes. Think of it as basic 
common sense. In fact, our Iowa Kids Count book is entitled "Reinventing Common Sense" to signify 
how remote our systems often seem from basic principles of how children and families succeed. 
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Setting Parameters 

If we want outcomes to be our friend, we must convincingly set out the parameters upon which our work 
should be judged. 

What I talked about is not the answer that elected officials want to hear. It is not a single program or 
service that lends itself to sound bites or produces impressive results before the next election. At the 
same time, however, although elected officials live in the electoral world, they also know that these are 
tough issues and they want solutions, even if they are complex and long-term. What they need is clear 
talk on what to expect and what it will take. 

We must make a case for "scale." Strategies must be adopted at a scale sufficient to influence outcomes 
on a community- wide basis. Returning to the conceptual framework I presented earlier, we must make 
the case that sufficient resources must be devoted to the strategies that, if they work, they will succeed 
with a large enough number of families and children to produce change in long-term community 
outcomes. 

We must not leave parts of the puzzle out of the picture. We must be clear of what we can, and cannot 
do, particularly if we are not going to address community disinvestment and seek to make villages 
whole. We will experience intervention "fade-out" if the context in which families take steps remain 
hostile to future gain. 

We must place appropriate expectations regarding timeframes. The Perry Pre-School Project has been 
widely regarded by policy makers as sound evidence that high quality pre-school programs are 
cost-effective. Yet our return on investment is a decade-and-a- half down the road. Patience is needed, if 
generational effects take a generation to occur. We must create realistic political expectations; you 
cannot force generational effects into a two-year election cycle window. 

We must present issues of cost in context . In Michigan over the last two decades, spending on housing 
prisoners has increased from 3 % to 14 % of state spending&shyp; approximately $1 billion increased 
share of the total budget. That would fund a lot of family support and community-building services 
which might have done far more to protect the safety and well-being of Michigan residents than housing 
35,000 more prisoners. 
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Ultimately, we must change our own efforts when they do not produce the intermediate or proximate 
outcomes that we believe they need to produce to achieve success. We do need to learn by doing, and if 
we take risks we will have failures. The challenge is not to institutionalize them but to learn from them 
to eventually succeed. 
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About Charles Bruner 

Charles Bruner serves as Executive Director of the Child and Family Policy Center, a nonprofit 
organization established in 1989 "to better link research and policy on issues vital to children and 
families." He holds an M.A. and Ph.D., in political science from Stanford University. 

Through the Child and Family Policy Center, Bruner provides technical assistance to states, 
communities, and foundations on child and family issues and heads the technical assistance activities of 
the federally-established National Center for Service Integration. Bruner also has served as a consultant 
to the Annie E. Casey Foundation in developing its Family-to-Family Initiative, its strategies for 
state-level systems reform, and its evaluation grants project. 

Bruner has written widely on public policy approaches to developing more comprehensive, 
community-based responses to children, family, and neighborhood needs. 

Charles Brunner 

Executive Director 

Child and Family Policy Center 

100 Court Ave, Suite 312 

DesMoines, I A 50309 

515/280-9027 
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Remarks by Discussants 

Mary Telesford, Federation of Families for Children’s Mental Health, Washington, 
DC. 

After hearing Dr. Bruner's remarks, I also like to re-emphasize the need for strength-based models of 
service delivery. Too long we have emphasized the deficits of our communities which so often 
reinforces the negative outcomes we've come to expect. 

Let me tell you about some of the strengths that impoverished communities do have, and that researchers 
need to recognize and work with. 

1 . Churches are community institutions that often know the history of what's happened in a 
neighborhood and have the ability to organize constituencies; 

2. Community-based organizations also have a tie to a community's history, proximity to the people 
and knowledge of culturally relevant issues; and 

3. Schools can offer access to many communities and families and can provide a neutral meeting 
ground. 

From my recent work (unclear reference) we've been able to observe the following pattern among 
women of color who live in impoverished circumstances and are successful. 

Foremost, they provide a spiritual base for their families and maintain a strong support system. They 
also maintain personal sobriety and do not use alcohol or illegal substances. They have meaningful 
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employment and value education, even if it has been delayed or interrupted. 

Clearly if we want to build successful communities, we need to build upon strength-based models and 
reach out to the people and families we serve. 

We need to understand what families want, which is: 

1 . to receive support when they need it, 

2. to have a single place for services&shyp;a one-stop shop, instead of the myriad service systems 
they face; and 

3. to receive recognition for their efforts. 

If we can help families and recognize their strengths, then the outcomes will take care of themselves. 

Cliff Davis 

Chief, Office of Children's Services and Prevention, Ohio Department of Mental Health 

I'd like to add, within the context that Dr. Bruner described, one piece that is important to our 
understanding outcomes achieved from the helping process. 

Regardless of the theories you have been trained in or the therapeutic tools you use with the persons who 
come to you for help, the only thing we know for sure is that the quality of the relationship between the 
person receiving or seeking help and the person offering help is a key to what kinds of outcomes are 
achieved. 

I am a therapist by trade, have spent much of my career as a community-based family therapist, and I 
know I did many good things&shyp;I know intuitively, instinctively&shyp;my wife could tell by 
looking at me whether the work I had done that day felt good to me. 

Regardless of the type of indicators we talk about, we are still a long way from being able to define or 
articulate this important relationship quality for people who are involved in the helping process. Until 
we are able to work together to understand, articulate, and measure (if we have to), that relationship and 
its components, we still aren't really going to understand what it is that we need to be training, and what 
it is we have to be doing in order to achieve positive, successful outcomes for our kids and their families. 

As we build processes to try to measure, understand, and articulate how we are affecting children and 
their families, we have to start with the families and front line workers. We have to work hand-in-hand 
with them throughout that process and ultimately the measure of our success will be determined by those 
families and front line workers. 

It does no good for me to sit in the state office tower in downtown Columbus, in central Ohio, and 
establish outcome measures that ultimately determine whether or not my system is successful. It has to 
start through focus group discussion, and implementation of participatory processes with families and 
front line workers; they have to be integrally related throughout the whole process and it is up to them to 
interpret the results. We need to create the mechanisms that allow this to happen. 

In the context of all the different community factors that Dr. Bruner described, asking families and 
workers to help us strikes me as one of the keys, at least in the mental health system, of the work that we 
are doing. I had compiled a list of real life kinds of indicators&shyp;I suspect everyone in this room 
could generate a similar kind of list. We need to pay attention to those things that our families, our kids 
and our front line workers say are the true indication of what our system is doing. 



About the Lecture Series 



The Gwen R. Iding Brogden Lecture Series was established as a lasting tribute to a citizen advocate. For 
more than 20 years, Ms. Iding Brogden championed the creation of high quality mental health services 
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in local, state and national arenas. The Lecture series features national speakers known for significant 
contributions in the area of children's mental health policy. For more information, contact Larry 
Schwartz at the Department of Child and Family Studies, Florida Mental Health Institute, University of 
South Florida, 8 1 3/974-464 1 . 
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Table 2 

"Whole Village" Keys to Child, Youth Success - 
Existence of Public Goods and Opportunities 

Stable environment for families, socially and economically 

• adequacy of housing stock 

• reasonableness of rent 

• mobility rates 

• employment opportunities 

• availability of general public goods (transportation, retail goods, recreation, etc.) 

Safe neighborhood and community 

• crime 

• environmental conditions (lead paint, abandoned buildings, etc.) 

Nurturing relationships with adults 

• parental participation in school life and activities adult participation in developing and supporting 
child and youth activities 

• places known by community as supportive learning environments (e.g. family centers) 

• array and supply of quality child care arrangements 

• schools as safe and supportive places for children 

Involvement in positive peer activities 

• participation levels in youth recreational activities, numbers and diversity 

• supervised play areas, parks, and events 

Diverse connections with community&shyp;school, church, neighborhood, civic activities 

• public library usage 

• church membership and activities 

• youth-based organizations 

• community recognition of youth in events and activities 

• voter registration and participation 

Realistic opportunities to realize! aspirations 

• business sponsorship of employment-education connections 

• diverse choices to excel and succeed in and out of school 

• positive climate and attitude among residents toward achieving potential 
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